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(57)Abstract: 

PROBLEM TO BE SOLVED: To provide an image 
pickup device that simultaneously photographs a moving 
picture and a still picture and obtains a still picture of 
high resolution. 

SOLUTION: A filter processing section 40 applies 
resolution conversion processing to an image pickup 
signal outputted from an image pickup section 20. A 
control section 80 controls operations of the image 
pickup section 20 and the filter processing section 40 so 
as to output an image pickup signal with higher 
resolution than the moving picture resolution for one 
frame period by each period which is an integer N (N>2) 
multiple of the frame period of the moving picture and so 
as to output the image pickup signal with the moving 
picture resolution for other frame period. The filter processing section 40 applies resolution 
conversion processing to convert the image pickup signal having the resolution higher than 
the moving picture resolution into the image pickup signal with the moving picture resolution 
for one frame period by each period of an integer multiple of the frame period of the moving 
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picture to obtain the high frequency component signal giving information with the resolution 
higher than the moving picture resolution and the image pickup signal with the moving picture 
resolution whose resolution is converted. 
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RECORD DEVICE, IMAGE RECORD/REPRODUCTION DEVICE, AND 
IMAGE TRANSMISSION METHOD 

(58) Abstract 

[Problem to be Solved] 

To shoot moving images and still images 
simultaneously, and obtain the still images of high 
resolution . 

[Solution] 

A resolution converting process is performed by a 
filter processing unit 40 on an image pickup signal 
output from an image pickup unit 20. A control unit 80 
controls the operations of the image pickup unit 20 and 
the filter processing unit 40 to output an image pickup 
signal of the resolution higher than moving image 
resolution in one frame period in each of the moving 
image frame periods of the multiples of an integer N (N 
> 2), and output an image pickup signal of moving image 
resolution in other frame periods. In one frame period 
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in each of the moving image frame periods of the 
multiple of the integer, the filter processing unit 40 
performs a resolution converting process of converting 
an image pickup signal of the resolution higher than 
moving image resolution into an image pickup signal of 
the moving image resolution, and obtains a signal of a 
high-frequency component for providing information 
about the resolution higher than moving image 
resolution and a resolution-converted image pickup 
signal of the moving image resolution. 
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[Claims for the Patent] 
[Claim 1] 

An image pickup device comprising: 

image pickup means for reading an image pickup 
signal in a predetermined reading period from an image 
pickup element and outputting the signal; 

resolution converting means for performing a 
resolution converting process on the image pickup 
signal output from said image pickup means; and 

control means for controlling operations of said 
image pickup means and resolution converting means to 
output an image pickup signal of the resolution higher 
than moving image resolution in one frame period in 
each of moving image frame periods of multiples of an 
integer N (N > 2), and output an image pickup signal of 
the resolution higher than moving image resolution in 
other frame periods, wherein 

in one frame period in each of the moving image 
frame periods of multiples of an integer N (N > 2), 
said resolution converting means performs a resolution 
converting process of converting an image pickup signal 
of the resolution higher than moving image resolution 
into an image pickup signal of the moving image 
resolution, and outputs a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
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resolution-converted image pickup signal of the moving 
image resolution. 
[Claim 2] 

The image pickup device according to claim 1, 
wherein : 

said image pickup means comprises drive means for 
driving the image pickup element in two operation modes, 
that is, a first operation mode in which an image 
pickup signal of a first resolution is read from the 
image pickup element, and a second operation mode in 
which an image pickup signal of a second resolution 
lower than the first resolution is read from the image 
pickup element; and 

said control means controls said drive means to 
be operated in the first operation mode in one frame 
period in each of the moving image frame periods of 
multiples of an integer N (N > 2), and in the second 
operation mode in other frame periods. 
[Claim 3] 

The image pickup device according to claim 1, 
further comprising: 

drive means for driving the image pickup element 
to read an image pickup signal of the resolution higher 
than moving image resolution from the image pickup 
element, wherein 

said control means controls the operations of 
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said resolution converting means to perform the 
resolution converting process of converting an image 
pickup signal, read from the image pickup element, of 
the resolution higher than moving image resolution into 
an image pickup signal of the moving image resolution, 
output a signal of a high-frequency component for 
providing information about the resolution higher than 
moving image resolution and a resolution-converted 
image pickup signal of moving image resolution in one 
frame period in each of the moving image frame periods 
of multiples of an integer N (N > 2), and output a 
resolution-converted image pickup signal of moving 
image resolution in other frame periods. 
[Claim 4] 

An image record device, comprising: 
resolution converting means for converting an 

image signal of the resolution higher than moving image 

resolution into an image pickup signal of the moving 

image resolution; 

compressing means for performing a compressing 

process on the image pickup signal of the moving image 

resolution; 

record means for recording in a recording medium 
the image signal of the moving image resolution on 
which said compressing means performs the compressing 
process; and 
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control means for controlling an operation of 
each of said means, wherein 

said resolution converting means performs a 
resolution converting process of converting an image 
signal of the resolution higher than moving image 
resolution into an image signal of the moving image 
resolution, and obtains a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
resolution-converted image pickup signal of the moving 
image resolution, and said record means records 
obtained signals in a recording medium in one frame 
period in each of the moving image frame periods of 
multiples of an integer N (N > 2) . 
[Claim 5] 

The image record device according to claim 4, 
further comprising: 

image pickup means comprising: an image pickup 
element from which an image pickup signal is read in a 
predetermined reading period; and drive means for 
driving said image pickup element in two operation 
modes, that is, a first operation mode in which an 
image pickup signal of a first resolution is read from 
said image pickup element, and a second operation mode 
in which an image pickup signal of a second resolution 
lower than the first resolution is read from said image 
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pickup element, wherein 

said control means controls said drive means to 
be operated in the first operation mode in one frame 
period in each of the moving image frame periods of 
multiples of an integer N (N > 2), and in the second 
operation mode in other frame periods, and controls an 
operation of said resolution converting means to 
convert the image pickup signal of a first resolution 
obtained by said image pickup means in the first 
operation mode into an image signal of the moving image 
resolution . 
[Claim 6] 

The image record device according to claim 4, 
further comprising: 

image pickup means comprising an image pickup 
element from which an image pickup signal of the 
resolution higher than moving image resolution, wherein 

said control means controls the operations of 
said resolution converting means to perform the 
resolution converting process of converting an image 
pickup signal, read from the image pickup element, of 
the resolution higher than moving image resolution into 
an image signal of the moving image resolution, output 
a signal of a high-frequency component for providing 
information about the resolution higher than moving 
image resolution and a resolution-converted image 
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signal of the moving image resolution in one frame 
period in each of the moving image frame periods of 
multiples of an integer N (N > 2), and output a 
resolution-converted image pickup signal of the moving 
image resolution in other frame periods. 
[Claim 7] 

The image record device according to claim 4, an 
image signal of the moving image resolution, a signal 
of a high-frequency component for providing information 
about the resolution higher than moving image 
resolution, and a resolution-converted image signal of 
the moving image resolution are recorded in a format in 
accordance with a format in which spatial resolution is 
scalable . 
[Claim 8] 

An image record/reproduction device, comprising: 

resolution converting means for converting an 
image signal of the resolution higher than moving image 
resolution into an image pickup signal of the moving 
image resolution; 

compressing/decompressing means for performing a 
compressing/decompressing process on the image pickup 
signal of the moving image resolution; 

record/reproduction means for 
recording/reproducing through a recording medium the 
image signal of the moving image resolution on which 
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said compressing means performs a compressing process; 
and 

control means for controlling an operation of 
each of said means, wherein: 

during recording, said resolution converting 
means performs a resolution converting process of 
converting an image signal of the resolution higher 
than moving image resolution into an image signal of 
the moving image resolution, and obtains a signal of a 
high-f reguency component for providing information 
about the resolution higher than moving image 
resolution and a resolution-converted image pickup 
signal of the moving image resolution, and said 
record/reproduction means records obtained signals in a 
recording medium in one frame period in each of the 
moving image frame periods of multiples of an integer N 
(N > 2) ; and 

during reproducing, said record/reproduction 
means reproduces from the recording medium a signal of 
a high-f reguency component for providing information 
about the resolution higher than moving image 
resolution and a resolution-converted image signal of 
the moving image resolution, and generates an image 
signal of a still image of the high resolution from the 
signal of the high-f reguency component and the 
resolution-converted image signal of the moving image 
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resolution . 
[Claim 9] 

An image transmission method of transmitting an 
image signal of moving image resolution, comprising: 

performing a resolution converting process of 
converting an image signal of the resolution higher 
than moving image resolution into an image signal of 
the moving image resolution, generating a signal of a 
high-frequency component for providing information 
about the resolution higher than moving image 
resolution and a resolution-converted image signal of 
the moving image resolution, and transmitting the 
signal of a high-frequency component for providing 
information about the resolution higher than moving 
image resolution together with the resolution-converted 
image signal of the moving image resolution in one 
frame period in each of the moving image frame periods 
of multiples of an integer N (N > 2) . 
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[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] 

The present invention relates to an image pickup 
device, an image record device, an image 
record/reproduction device, and an image transmission 
method for recording/reproducing a digital video signal 
captured using an image pickup element as moving images 
or a still image through a recording medium. 

[0002] 

[Conventional Art] 

Conventionally, there has been the development of 
a digital video camera capable of compressing a digital 
video signal by a discrete cosine transform (DCT) , a 
wavelet transformation, and a variable length code 
based on an image pickup signal captured using an image 
pickup element, and recording a result on a recording 
medium such as a magnetic tape, a magnetic disk, an 
optical disk, etc. For the digital video camera, not 
only a moving image recording mode but also a still 
image recording mode is prepared. In the still image 
recording mode of a digital video camera, a still image 
is compressed and recorded on a recording medium. In 
the moving image recording mode, moving images are 
compressed and recorded on a recording medium. When 
the still image recording mode is set, an image signal 
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of one frame is recorded repeatedly for five seconds. 
In the still image recording mode, a still image can be 
recorded as digital data. Therefore, it can be used 
when data is fetched to a personal computer for 
processing, and when the data is directly printed out. 
[0003] 

In this example, with the image signal, spectra 
are concentrated normally on a low frequency area, and 
the spectra decrease toward higher frequencies. In the 
DCT and wavelet transformation, based on the above- 
mentioned characteristic, the spectrum of each base is 
biased by dividing a signal for each band by an 
orthogonal transform. Then, the data biased in the 
spectrum of each base is quantized, and a symbol having 
a high probability of occurrence is assigned a short 
code, and a symbol having a low probability of 
occurrence is assigned a long code, thereby compressing 
data . 
[0004] 

[Problems to be Solved by the Invention] 

In the conventional digital video camera, when a 
frame of moving image is retrieved as a JPEG still 
image, only a still image as good as moving image 
resolution (720 pixels x 484 pixels) can be obtained. 
[0005] 

In a digital video camera capable of shooting a 
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still image of higher resolution, when moving images 
are shot (about television monitor resolution) and a 
still image is shot, a user has to necessarily select 
which is to be shot. That is, moving images and a 
still image cannot be simultaneously shot. 
[0006] 

An object of the present invention is to provide 
an image pickup device, an image record device, an 
image record/reproduction device, and an image 
transmission method capable of simultaneously shooting 
moving images and a still image, and obtaining a still 
image of the high resolution. 
[0007] 

[Means for Solving the Problems] 

An image pickup device according to the present 
invention includes: image pickup means for reading an 
image pickup signal in a predetermined reading period 
from an image pickup element and outputting the signal; 
resolution converting means for performing a resolution 
converting process on the image pickup signal output 
from the image pickup means; and control means for 
controlling operations of the image pickup means and 
resolution converting means to output an image pickup 
signal of the resolution higher than moving image 
resolution in one frame period in each of moving image 
frame periods of multiples of an integer N (N > 2), and 
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output an image pickup signal of the resolution higher 
than moving image resolution in other frame periods . 
In one frame period in each of the moving image frame 
periods of multiples of an integer N (N > 2), the 
resolution converting means performs a resolution 
converting process of converting an image pickup signal 
of the resolution higher than moving image resolution 
into an image pickup signal of the moving image 
resolution, and outputs a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
resolution-converted image pickup signal of the moving 
image resolution. 
[0008] 

An image record device according to the present 
invention includes: resolution converting means for 
converting an image signal of the resolution higher 
than moving image resolution into an image pickup 
signal of the moving image resolution; compressing 
means for performing a compressing process on the image 
pickup signal of the moving image resolution; record 
means for recording in a recording medium the image 
signal of the moving image resolution on which the 
compressing means performs a compressing process; and 
control means for controlling an operation of each of 
the means. The resolution converting means performs a 
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resolution converting process of converting an image 
signal of the resolution higher than moving image 
resolution into an image signal of the moving image 
resolution, and obtains a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
resolution-converted image pickup signal of the moving 
image resolution, and the record means records obtained 
signals in a recording medium in one frame period in 
each of the moving image frame periods of multiples of 
an integer N (N > 2) . 
[0009] 

An image record/reproduction device includes: 
resolution converting means for converting an image 
signal of the resolution higher than moving image 
resolution into an image pickup signal of the moving 
image resolution; compressing/decompressing means for 
performing a compressing/decompressing process on the 
image pickup signal of the moving image resolution; 
record/reproduction means for recording/reproducing 
through a recording medium the image signal of the 
moving image resolution on which the compressing means 
performs a compressing process; and control means for 
controlling an operation of each of the means. During 
recording, the resolution converting means performs a 
resolution converting process of converting an image 
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signal of the resolution higher than moving image 
resolution into an image signal of the moving image 
resolution, and obtains a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
resolution-converted image pickup signal of the moving 
image resolution, and the record/reproduction means 
records obtained signals in a recording medium in one 
frame period in each of the moving image frame periods 
of multiples of an integer N (N > 2) . During 
reproducing, the record/reproduction means reproduces 
from the recording medium a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
resolution-converted image signal of the moving image 
resolution, and generates an image signal of a still 
image of the high resolution from the signal of the 
high-frequency component and the resolution-converted 
image signal of the moving image resolution. 
[0010] 

Furthermore, the present invention is an image 
transmission method of transmitting an image signal of 
the moving image resolution including: performing a 
resolution converting process of converting an image 
signal of the resolution higher than moving image 
resolution into an image signal of the moving image 
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resolution, generating a signal of a high-frequency 
component for providing information about the 
resolution higher than moving image resolution and a 
resolution-converted image signal of the moving image 
resolution, and transmitting the signal of a high- 
frequency component for providing information about the 
resolution higher than moving image resolution together 
with the resolution-converted image signal of the 
moving image resolution in one frame period in each of 
the moving image frame periods of multiples of an 
integer N (N > 2) . 
[0011] 

[Embodiments of the Invention] 

The embodiments of the present invention will be 
described below in detail with reference to the 
attached drawings. 

[0012] 

The present invention is applied to an image 
record/reproduction system 10 having the configuration 
as shown in Figure 1, for example. 
[0013] 

The image record/reproduction system 100 includes 
an image pickup unit 20, a storage unit 30, a filter 
processing unit 40, an image compressing/decompressing 
unit 50, a record/reproduction unit 60, a video output 
unit 70, a control unit 80 for controlling the 
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components, etc. connected via a data bus 10. 
[0014] 

The control unit 80 includes a system controller 
81 using a microcomputer, accepts pressing operation 
input of a shutter button 82, and performs control of a 
recording operation mode. That is, the image 
record/reproduction system 100 starts a recording 
operation using the pressing operation of the shutter 
button 82 as a trigger, and performs control of the 
recording operation mode by the system controller 81 
detecting pressing operation input of the shutter 
button 82. 
[0015] 

The image pickup unit 20 includes a solid-state 
image pickup element 22 such as a CCD image sensor 
driven by a drive circuit 21. An image pickup signal 
obtained by the solid-state image pickup element 22 is 
provided for an image pickup signal processing circuit 
23. The image pickup signal processing circuit 23 
converts an image pickup signal obtained by the solid- 
state image pickup element 22 into a brightness signal 
and a color difference signal. 
[0016] 

In the image record/reproduction system 100, the 
drive circuit 21 drives the solid-state image pickup 
element 22 by switching two types of operation mode, 
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that is, the first operation mode in which a high- 
resolution image pickup signal is read from the solid- 
state image pickup element 22 and the second operation 
mode in which a low-resolution image pickup signal is 
read . 
[0017] 

In the first operation mode, as shown by (A) and 
(B) in Figure 2, the solid-state image pickup element 
22 is driven to read all pixel signals by dividing them 
into two fields, thereby reading a high-resolution (for 
example, 2560 x 1920 pixels) image pickup signal from 
the solid-state image pickup element 22. In addition, 
in the second operation mode, as shown in Figure 3, the 
solid-state image pickup element 22 is driven to output 
signals of all effective area in the 1/4 period of the 
high resolution mode by reading two lines in 8 lines, 
thereby reading a low-resolution (for example, 2560 x 
480 pixels) image pickup signal from the solid-state 
image pickup element 22. Thus, in the image pickup 
unit 20 in the image record/reproduction system 100, 
the solid-state image pickup element 22 is driven in 
the first operation mode only in one frame in the N 
frame period, and the solid-state image pickup element 
22 is driven in the second operation mode in other (N- 
1) frames, thereby obtaining a low-resolution image 
pickup signal as moving image pickup image output as 
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shown in Figure 4, and obtaining a high-resolution 
image pickup signal as still image pickup image output 
only in one frame in each of N frames. 
[0018] 

The high-resolution image pickup signal obtained 
by driving the solid-state image pickup element 22 in 
the first operation mode is once stored in the storage 
unit 20. From the high-resolution image pickup signal 
stored in the storage unit 20, a low-resolution image 

(640 x 480 pixels) 1/4 band restricted vertically and 
horizontally by the filter processing unit 40 is 
generated. The low-resolution image generated by the 
filter processing unit 40 is image-compressed by the 
image compressing/decompressing unit 50 in the MPEG-2, 
the DV format, etc., and recorded as a low resolution 
moving image stream on a recording medium by the 
record/reproduction unit 60. Simultaneously, the high- 
freguency component removed from the high-resolution 
image pickup signal when the low-resolution image is 
generated is recoded on a recording medium as metadata 
of the low resolution moving image stream by the 
record/reproduction unit 60. 

[0019] 

In addition, the low-resolution image pickup 
signal obtained by driving the solid-state image pickup 
element 22 in the second operation mode is once stored 
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in the storage unit 20. From the low-resolution image 
pickup signal stored in the storage unit 20, a low- 
resolution image (640 x 480 pixels) 1/4 band restricted 
horizontally by the filter processing unit 40 is 
generated. The low-resolution image generated by the 
filter processing unit 40 is image-compressed by the 
image compressing/decompressing unit 50 in the MPEG-2, 
the DV format, etc., and recorded as a low resolution 
moving image stream on a recording medium by the 
record/reproduction unit 60. 
[0020] 

That is, when the system controller 81 in the 
image record/reproduction system 100 enters the control 
of the recording operation mode by detecting pressing 
operation input of the shutter button 82, it first 
performs an initializing process by setting the frame 
number n to 0 as shown by the flowchart in Figure 5, 
and setting the period, in which a high-resolution 
image pickup signal obtained by driving the solid-state 
image pickup element 22 in the first operation mode is 
processed, to N (N > 2) (step SI) . 
[0021] 

It is determined whether or not an image pickup 
operation is being performed (step S2). 
[0022] 

If the determination result in step S2 is YES, 
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that is, when the image pickup operation is being 
performed, it is determined whether or not Frame%N==0? 
(where % indicates a remainder operator), that is, 
whether or not the remainder obtained by dividing the 
frame number of the image pickup signal to be processed 
by the period N is 0 (step S3) . If the determination 
result in step S2 is NO, that is, when the image pickup 
operation is not being performed, then the control of 
the recording operation mode is terminated. 
[0023] 

If the determination result in step S3 is YES, 
that is, the remainder is 0, then a high-resolution 
image pickup signal is obtained by driving the solid- 
state image pickup element 22 in the first operation 
mode, thereby performing high resolution processing 
(step S4) . If the determination result in step S3 is 
NO, that is, the remainder is not 0, then the solid- 
state image pickup element 22 is driven in the second 
operation mode to obtain a low-resolution image pickup 
signal, thereby performing low resolution processing 
(step S5) . 
[0024] 

In the high resolution processing in step S4, the 
operation mode of the drive circuit 21 of the image 
pickup unit 20 is first switched to the first operation 
mode as shown by the flowchart in Figure 6 (step S41) . 
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[0025] 

Next, the high resolution image (2560 x 1920 
pixels) obtained by driving the solid-state image 
pickup element 22 in the first operation mode is 
divided vertically and horizontally by the filter 
processing unit 40 into low frequency components and 
high frequency components, that is, four band 
components of a low frequency component (LL) , a 
vertical high frequency component (LH) , a horizontal 
high frequency component (HL) , and a diagonal high 
frequency component (HH) as shown in Figure 7 (step 
S42) . 
[0026] 

Then, the vertical high frequency component (LH) , 
the horizontal high frequency component (HL) , and the 
diagonal high frequency component (HH) obtained in the 
band dividing process in step S42 are stored as 
metadata on a recording medium by the image 
record/reproduction unit 60 (step S43) . 
[0027] 

The low frequency component (LL) obtained in the 
band dividing process in step S42 is divided by the 
filter processing unit 40 vertically and horizontally 
into low frequency components and high frequency 
components, that is, four band components of a low 
frequency component (LLLL) , a vertical high frequency 
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component (LLLH) , a horizontal high frequency component 
(LLHL) , and a diagonal high frequency component (LLHH) 
as shown in Figure 7 (step S44) . 
[0028] 

Then, the vertical high frequency component 
(LLLH) , the horizontal high frequency component (LLHL) , 
and the diagonal high frequency component (LLHH) 
obtained in the band dividing process in step S44 are 
stored as metadata on a recording medium by the image 
record/reproduction unit 60 (step S45) . 
[0029] 

The image (640 x 480 pixels) of the low frequency 
component (LLLL) obtained in the band dividing process 
in step S44 is image-compressed by the image 
compressing/decompressing unit 50 in the MPEG-2, the DV 
format, etc. (step S46) . 
[0030] 

The compressed data of the low-resolution image 
obtained in the image compressing process in step S46 
is stored as a low resolution moving image stream on a 
recording medium by the image record/reproduction unit 
60 (step S47) . 
[0031] 

In the low resolution processing in step S5, as 
shown by the flowchart in Figure 8, the operation mode 
of the drive circuit 21 of the image pickup unit 20 is 
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first switched to the second operation mode (step S51). 
[0032] 

The horizontal direction of the low-resolution 
image (2560 x 480 pixels) obtained by driving the 
solid-state image pickup element 22 in the second 
operation mode is band-restricted into 1/4 frequency by 
the filter processing unit 40 (step S52), thereby 
resampling the horizontal direction into 1/4 (step S53) . 
[0033] 

The obtained low-resolution image (640 x 480 
pixels) is image-compressed by the image 
compressing/decompressing unit 50 in the MPEG-2 and the 
DV format (step S54) in the MPEG-2, the DV format, etc. 
(step S54) . 
[0034] 

The compressed data of the low-resolution image 
obtained in the image compressing process in step S54 
is recorded as a low resolution moving image stream on 
a recording medium by the image record/reproduction 
unit 60 (step S55) . 
[0035] 

Thus, the moving images and still image recorded 
on the recording medium are reproduced as follows. 
[0036] 

That is, in the image record/reproduction system 
100, in the moving image reproducing operation, in the 
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compressed stream retrieved from the recording medium 
by the image record/reproduction unit 60, the image 
compressing/decompressing unit 50 performs a decoding 
process corresponding to the MPEG-2 and the DV format 
on the portion corresponding to the moving image data, 
and outputs a video signal through the video output 
unit 70. 
[0037] 

When a still image is reproduced, a compressed 
portion of the low frequency component of a high 
resolution image as a source of a still image is 
retrieved from the compressed stream retrieved from the 
recording medium by the image record/reproduction unit 
60, and the image compressing/decompressing unit 50 
performs a decoding process corresponding to the MPEG-2 
and the DV format, and attaches a decompressed low- 
resolution image to the storage unit 20, and outputs it 
as a video signal through the video output unit 70. 
The metadata of the high frequency portion is retrieved 
from the recording medium by the image 
record/reproduction unit 60, a predetermined 
reconstructing process is performed, and a result is 
attached to the storage unit 20. Thus, a high 
resolution image is reconstructed by the filter 
processing unit 40 from the image data attached to the 
storage unit 20. The image compressing/decompressing 
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unit 50 performs appropriate image processing on the 
reconstructed high resolution image, for example, a 
compressing process such as the JPEG compression etc. 
is performed, and the image record/reproduction unit 60 
records a still image on a recording medium. 
[0038] 

The reconstructed high resolution image can also 
be stored on a recording medium using a recording 
device different from the image record/reproduction 
unit 60 for recording the moving image and still image 
streams on the recording medium. 
[0039] 

In the image record/reproduction system 100, a 
user can retrieve a desired scene as a higher- 
resolution still image while storing moving images 
libraries as if the user were using a camcoder. 
[0040] 

In the image record/reproduction system 100, the 
image pickup unit 20 drives the solid-state image 
pickup element 22 in the first operation mode for one 
frame in the N frame period, and drives the solid-state 
image pickup element 22 in the second operation mode in 
the other (N-l) frames, thereby obtaining a low- 
resolution image pickup signal as image pickup output 
of moving images, and simultaneously obtaining a high- 
resolution image pickup signal as image pickup output 
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of a still image for only one frame in the N frames. 
It is also possible to generate a low-resolution image 
for moving images by the filter processing unit 40 by 
constantly reading a high-resolution image pickup 
signal for a still image from the solid-state image 
pickup element 22 of the image pickup unit 20. 
[0041] 

In the image record/reproduction system 100, the 
low-resolution image for moving images is recorded in a 
predetermined format, for example, the JPEG, DV, MPEG, 
etc., and a high frequency component of a high 
resolution image for a still image is recorded as 
metadata. However, as shown by each flowchart in 
Figures 9 and 10, all data of low resolution and high 
resolution can be recorded in accordance with a format 
in which the spatial resolution is scalable as in the 
JPEG-2000 by replacing the processes in steps S42 to 
S46, and S54 with the processes in steps S42 ' to S46' 
and S54 ' . 
[0042] 

In the JPEG-2000, as an image compressing method, 
subband coding by the wavelet transformation capable of 
concentrating most energy of an image on a small 
portion of a sample while maintaining the entire energy 
is adopted. In the subband coding, a digital signal is 
band-divided by a filter for performing a wavelet 
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transformation to compress the digital signal. A 
filtering process is performed using a plurality of 
filters having different pass bands, and then down- 
sampling is performed at an interval depending on each 
frequency band, and a compressing process is performed 
using the bias of energy of an output signal of each 
filter. The signal processing is performed by the band 
division using the subband coding and the wavelet 
transformation described in, for example, the document 
"Wavelet Transformation and Subband Coding" by Martin 
Vetari, the Institute of Electronics, Information and 
Communication Engineers, Vol. 1.74, No. 12, pp 1275- 
1278 in December 1991. 
[0043] 

Figures 11(A) and 11(B) show examples of 
coefficients of a low pass filter (LPF) and a high pass 
filter (HPF) used as a filter for performing a wavelet 
transformation. Figure 12 shows the filter 
characteristics of the filters. In the wavelet 
transformation, the filters are applied vertically and 
horizontally with respect to an image, the band is 
restricted, and samples are thinned into 1/2. As shown 
in the schematic diagram in Figure 13, the band is 
divided into four band components, that is, the low 
frequency component (LL) , the vertical high frequency 
component (LH) , the horizontal high frequency component 
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(HL) , and the diagonal high freguency component (HH) , 
in the first process. The low freguency component (LL) 
is obtained using the LPF horizontally and vertically. 
The vertical high freguency component (LH) is obtained 
using the LPF horizontally and the HPF vertically. The 
horizontal high freguency component (HL) is obtained 
using the HPF horizontally and the LPF vertically. The 
diagonal high freguency component (HH) is obtained 
using the HPF horizontally and vertically. In the 
wavelet transformation, the filters are similarly used 
for the low freguency component (LL) , and the similar 
processes are performed recursively. 
[0044] 

Furthermore, in the above-mentioned image 
record/reproduction system 100, the streams of moving 
images and still images are recorded and reproduced 
through a recording medium by the image 
record/reproduction unit 60. It is also possible to 
replace the image record/reproduction unit 60 with 
communication means to apply the present invention to 
an image transmitting system for transmitting moving 
images and still images through a transmitting system. 
[0045] 

[Advantages of the Invention] 

As described above, according to the present 
invention, moving images and still images are 
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simultaneously shot, and a still image of the high 
resolution can be obtained. Therefore, a user can 
retrieve a desired scene as a higher-resolution still 
image while storing moving images libraries as if the 
user were using a camcoder. According to the present 
invention, still images and moving images of double or 
higher resolution can be simultaneously shot in a 
period of multiples of an integer being 2 or higher of 
a moving image frame period. 

[Brief Description of the Drawings] 
[Figure 1] 

Figure 1 shows the configuration of the image 
record/reproduction system according to the present 
invention . 
[Figure 2] 

Figure 2 is an explanatory view of the operation 
in a first operation mode in which a high-resolution 
image pickup signal is read from a solid-state image 
pickup element of an image pickup unit in the image 
record/reproduction system. 
[Figure 3] 

Figure 3 is an explanatory view of the operation 
in a second operation mode in which a low-resolution 
image pickup signal is read from the solid-state image 
pickup element of the image pickup unit. 
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[Figure 4] 

Figure 4 shows the outline of a resolution of the 
image pickup signal read from the solid-state image 
pickup element of the image pickup unit. 
[Figure 5] 

Figure 5 is a flowchart of the control contents 
of a recording operation mode by a system controller in 
the image record/reproduction system. 
[Figure 6] 

Figure 6 is a flowchart of the procedure of the 
high resolution processing in the image 
record/reproduction system. 
[Figure 7] 

Figure 7 shows the outline of the resolution of a 
resolution-converted image in the image 
record/reproduction system. 
[Figure 8] 

Figure 8 is a flowchart of the procedure of the 
low resolution processing in the image 
record/reproduction system. 
[Figure 9] 

Figure 9 is a flowchart of the procedure of the 
process performed when a high resolution processing is 
performed in accordance with the JPEG-2000 in the image 
record/reproduction system. 
[Figure 10] 
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Figure 10 is a flowchart of the procedure of the 
process performed when a low resolution processing is 
performed in accordance with the JPEG-2000 in the image 
record/reproduction system. 
[Figure 11] 

Figure 11 shows an example of coefficients of the 
LPF and the HPF used as filters in performing the 
wavelet transformation. 
[Figure 12] 

Figure 12 shows the filter characteristics of the 
LPF and the HPF. 
[Figure 13] 

Figure 13 shows the outline of the resolution of 
a resolution-converted image by the wavelet 
transformation. 

[Description of Symbols] 

10 data bus, 20 image pickup unit, 21 drive 
circuit, 22 solid-state image pickup element, 23 
image pickup signal processing circuit, 30 storage 
unit, 40 filter processing unit, 50 image 
compressing/decompressing unit, 60 image 
record/reproduction unit, 70 video output unit, 80 
control unit, 81 system controller, 82 shutter button, 
100 image record/reproduction system 



- 33 - 



JPA 2002-044531 



Figure 1 

21 DRIVE UNIT 

2 2 SOLID-STATE IMAGE PICKUP ELEMENT 

2 3 IMAGE PICKUP SIGNAL PROCESSING CIRCUIT 

3 0 STORAGE UNIT 

40 FILTER PROCESSING UNIT 

50 IMAGE COMPRESSING/DECOMPRESSING UNIT 
60 IMAGE RECORD/REPRODUCTION UNIT 

7 0 VIDEO OUTPUT UNIT 

81 SYSTEM CONTROLLER 

82 SHUTTER BUTTON 

Figure 2 

#1 FIRST FIELD 
#2 SECOND FIELD 

Figure 4 

#1 MOVING IMAGE DATA 
#2 STILL IMAGE DATA 

Figure 5 
#1 START 

51 INITIALIZE 
PERIOD 

52 SHOOTING? 

S4 HIGH RESOLUTION PROCESSING 
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S5 LOW RESOLUTION PROCESSING 
#2 END 

Figure 6 

#1 HIGH RESOLUTION PROCESSING 

541 SWITCH DRIVE UNIT TO FIRST OPERATION MODE 

542 BAND-DIVIDE HIGH RESOLUTION IMAGE READ FROM 
SOLID-STATE IMAGE PICKUP ELEMENT 

543 RECORD HIGH FREQUENCY COMPONENT OBTAINED BY 
DIVIDING BAND AS METADATA 

S4 4 BAND-DIVIDE AGAIN LOW FREQUENCY COMPONENT 

OBTAINED BY DIVIDING BAND 
S45 RECORD HIGH FREQUENCY COMPONENT OBTAINED BY 

DIVIDING BAND AS METADATA 
S4 6 IMAGE-COMPRESS LOW FREQUENCY COMPONENT OBTAINED 

BY DIVIDING BAND AS METADATA 
S47 RECORD COMPRESSED DATA 
#2 RETURN 

Figure 7 

#1 FREQUENCY AREA OF IMAGE 
#2 HORIZONTAL FREQUENCY 

#3 VERTICAL FREQUENCY 

Figure 8 

#1 LOW RESOLUTION PROCESSING 
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551 SWITCH DRIVE UNIT TO SECOND OPERATION MODE 

552 BAND-RESTRICT THE HORIZONTAL DIRECTION OF LOW- 
RESOLUTION IMAGE READ FROM SOLID-STATE IMAGE 
PICKUP ELEMENT INTO 1/4 FREQUENCY 

553 RESAMPLE THE HORIZONTAL DIRECTION INTO 1/4 
S5 4 COMPRESS RESAMPLED IMAGE 

S55 RECORD COMPRESSED DATA 
#2 RETURN 

Figure 9 

#1 HIGH RESOLUTION PROCESSING 

S41 SWITCH DRIVE UNIT TO FIRST OPERATION MODE 
S42' BAND-DIVIDE HIGH RESOLUTION IMAGE READ FROM 

SOLID-STATE IMAGE PICKUP ELEMENT USING WAVELET 

FILTER 

S43' PERFORM COMPRESSING PROCESS IN ACCORDANCE WITH 

JPEG-2000 ON HIGH FREQUENCY COMPONENT OBTAINED BY 

DIVIDING BAND TO RECORD 
S44' BAND-DIVIDE AGAIN LOW FREQUENCY COMPONENT 

OBTAINED IN DIVIDING BAND BY WAVELET FILTER 
S45' PERFORM COMPRESSING PROCESS IN ACCORDANCE WITH 

JPEG-2000 ON HIGH FREQUENCY COMPONENT OBTAINED BY 

DIVIDING BAND TO RECORD 
S46' PERFORM COMPRESSING PROCESS IN ACCORDANCE WITH 

JPEG-2 000 ON LOW FREQUENCY COMPONENT OBTAINED BY 

DIVIDING BAND TO RECORD 
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S47 RECORD COMPRESSED DATA 
#2 RETURN 

Figure 10 

#1 LOW RESOLUTION PROCESSING 

551 SWITCH DRIVE UNIT TO SECOND OPERATION MODE 

552 BAND-RESTRICT THE HORIZONTAL DIRECTION OF LOW- 
RESOLUTION IMAGE READ FROM SOLID-STATE IMAGE 
PICKUP ELEMENT INTO 1/4 FREQUENCY 

553 RESAMPLE THE HORIZONTAL DIRECTION INTO 1/4 

S5 4' COMPRESS RESAMPLED IMAGE IN FORMAT IN ACCORDANCE 

WITH JPEG-2000 
S55 RECORD COMPRESSED DATA 
#2 RETURN 

Figure 11 

#1 COEFFICIENT OF LPF 
#2 COEFFICIENT OF HPF 
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